1. Field of the Invention
The present invention relates generally to color cameras, and more particularly to a single CCD type color camera having a color separation circuit for processing a signal from a solid state imaging device which includes photoelectric conversion elements arranged in an array corresponding to color filters arranged in a color differentially ordered array, and a method of separating color signals using such a circuit.
2. Description of the Background Art
A CCD (Charge Coupled Device) which is widely used today as an imaging device in a color camera simply changes the amplitude of a signal to output in response to the brightness of light which it receives, and the output signal does not include any color information. Therefore, in order to obtain color information, some optical means must be employed for example for filtering light incident to the CCD.
For home use, a so-called single CCD type color camera which extracts three primary color signals from a single CCD is employed, and a color filter array is used on the side of the light receiving surface of the CCD according to a so-called simultaneous color imaging method.
Structure of Inter-line Transfer CCD
FIG. 16 is a block diagram schematically showing the structure of an inter-line transfer CCD 10 generally used in a color camera for home use.
Inter-line transfer CCD 10 includes a photosensitive portion 12 formed of p-n junction type photodiodes arranged in an array, a transfer portion 14 including an analog shift register formed of a CCD, and a horizontal transfer register 16 for transferring signals formed by converting signal charges sequentially transferred from transfer portion 14 into voltage in the horizontal direction for output.
FIG. 16 includes p-n junction type photodiodes corresponding to three pixels arranged each in the vertical and horizontal direction for ease of illustration. In an actual CCD for use in a color camera, photodiodes corresponding to 500 pixels in the vertical direction and 500 to 800 pixels in the horizontal direction for example are arranged in an array.
The operation will be briefly described.
When light comes into the photodiodes, charges are generated and accumulated within the diodes. Then, application of a prescribed voltage to a shift gate causes the accumulated charges to be simultaneously transferred to the analog shift register. The CCD analog shift register sequentially transfers charges toward horizontal transfer register 16 in response to applied clock pulse voltages .phi.V1, .phi.V2, and .phi.V3. Horizontal transfer register 16 converts thus transferred received signal charges into voltages, and is then driven by externally applied horizontal driving signals .phi.H1, .phi.H2, and .phi.H3 to externally sequentially output the voltages as imaging signal outputs.
Driving Method of Inter-line Transfer CCD
The inter-line transfer CCD is generally driven in two modes, i.e., frame accumulation and field accumulation. The field accumulation mode used corresponding to a color differentially ordered color filter array will be detailed.
FIGS. 17A and 17B are schematic diagrams for use in illustration of how signal charges are read out form inter-line transfer CCD 10 according to the field accumulation mode, FIG. 17A shows a method of reading out a signal charge in an odd-numbered field, and FIG. 17B a method of reading out a signal charge in an even-numbered field.
As shown in FIG. 17A, in an odd-numbered field, signals from an odd-numbered pixel and an even-numbered pixel in the vertical direction are simultaneously transferred from the photosensitive portion to the transfer portion, where the signals are added.
In an even-numbered field as shown in FIG. 17B, signals from an even-numbered pixel and an odd-numbered pixel in a different combination from the above are simultaneously transferred to transfer portion 14, where they are added.
If the color filter array is color differentially ordered, color differential signals are separated using such a method of transferring in inter-line transfer CCD 10.
Method of Separating Color Differential Signals
FIG. 18 is a diagram schematically showing the flow of processing a signal output from inter-line transfer CCD 10 when the color filter array is color differentially ordered.
As shown in FIG. 18, in the color differentially ordered color filter array, color filters of magenta (hereinafter Mg), green (hereinafter G), cyan (hereinafter Cy), and yellow (hereinafter Ye) are arranged in a mosaic.
Now, since so-called additive color process is possible in mixing colors in light, between three primary colors red (R), green (G) and blue (B) and the complementary colors Mg, Ye and Cy, the following relation is established: EQU Mg=R+B (1)
Ye=R+G (2) EQU Cy=B+G (3)
Therefore, using Mg, G, Ye and Cy described above as the colors of color filters permits the intensity of a G signal weighing the most in a luminance signal among three primary colors R, G and B to be higher than that of a B signal.
In the example shown in FIG. 18, an array portion of four rows and four columns is extracted from the color differentially ordered color filter array for illustration. In odd-numbered rows (y-direction), color filters of Mg and color filters of G are alternately arranged in the horizontal direction (x-direction).
In even-numbered rows, color filters of Ye and color filters of Cy are alternately arranged in the horizontal direction. The method of reading out from inter-line transfer CCD 10 having the color filter array arrangement is on the basis of 2-pixel addition reading in the vertical direction (y-direction).
In an odd-numbered field, an odd-numbered pixel and an even-numbered pixel in the vertical direction are added, and in the following even-numbered field, a different combination of an even-numbered pixel and an odd-numbered pixel are added. Thus, in the even-numbered field for example signals are output in the order of G+Cy, Mg+Ye, . . . in the n-th scanning line and in the order of Mg+Cy, G+Ye, . . . in the n+1-th scanning line.
These signals are pulse amplitude modulated signals as shown in FIG. 18. In FIG. 18, based on the above expressions (1) to (3), the complementary color signals are replaced by three primary color signals and indicated as the amplitude modulated waveforms of the three primary color signals.
A direct current component and a fundamental wave component excluding a harmonic component are shown as follows.
Signal So in the n-th scanning line in the even-numbered field is as follows: EQU So=(Mg+Ye)+(G+Cy)+1/2.multidot.{(Mg+Ye)-(G+Cy)}sin(2.pi.fnt)=2R+3G+2B+1/ 2.multidot.(2R-G)sin(2.pi.fnt) (4)
Signal Se in the n+1-th scanning line is as follows: EQU Se=(Mg+Cy)+(G+Ye)+1/2.multidot.{(Mg+Cy)-G+Ye)}sin(2.pi.fnt)=2R+3G+2B+1/ 2.multidot.(2B-G)sin(2.pi.fnt) (5)
where fn indicates a Nyquist frequency, i.e. a sampling frequency.
A luminance signal is obtained by filtering only the direct current component in the above expressions (4) and (5) through a low-pass filter (hereinafter LPF).
The luminance signal and two color differential signals 2R-G and 2B-G can be readily separated if passed through a band-pass filter (hereinafter BPF) having a center frequency fn and detected.
More specifically, the luminance signal and color differential signals can be line-sequentially obtained.
However, in view of a single scanning line, only the color differential signals obtained are 2R-G and 2B-G, and the color signals corresponding to the scanning line cannot be reproduced.
The luminance signal and color signals are actually reproduced as follows:
A luminance signal CL is produced by the sum of adjacent pixels in each row in an output signal from the CCD: EQU CL=Ye+Cy+Mg+G=2R+3G+2B
The LPF with this end is implemented for example by a circuit shown in FIG. 19.
An LPF 20 includes a one-pixel delay circuit 22 receiving an output signal from the CCD, an adder circuit 24 for adding the output signal received from the CCD and the signal received from one-pixel delay circuit 22 for output, and an attenuator 26 for reducing the intensity of a signal by half.
The transfer function LPF(z) and frequency characteristic of the LPF are as follows: EQU LPF(Z)=(1+Z.multidot.1)/2 (6) EQU .vertline.LPF(Z).vertline.=.vertline.cos(.pi.f/2fn).vertline. (7)
The intensity of a Y signal is therefore gradually reduced from the low frequency region and reaches 0 at Nyquist frequency fn as shown in FIG. 20.
If CCD 10 has H effective pixels in the horizontal direction and V effective pixels in the vertical direction, the horizontal resolution for a luminance signal within a single field will be H and the vertical resolution will be V/2. More specifically, the resolution in the vertical direction is half the number of pixels arranged in the CCD.
Now, a separation processing for color signals will be described.
FIG. 21 is a block diagram schematically showing the structure of a conventional color separation circuit 500 which separates three primary color signals R, G, and B from a CCD output signal.
Color separation circuit 500 includes a sample-hold circuit 502 performing a sample and hold (S&H) operative on received CCD output signal in response to a sampling pulse SP1 and outputting a signal S1 (Ye+Mg, . . . in the n-th line), a sample-hold circuit 504 for performing a sampling and holding to a received CCD output signal in response a sampling pulse SP2 in a phase 180 different from that of sampling pulse SP1 and outputting a signal S2 (Cy+G, . . . in the n-th line), an adder circuit 506 for adding received signals S1 and S2 and outputting an added signal S1+S2, and a subtracter circuit 508 for outputting a differential signal S1-S2 between received signals S1 and S2.
Color separation circuit 500 further includes a 1H delay line 510 for delaying received signal S1-S2 by one horizontal scanning interval for output, a selecting circuit 512 for selectively outputting one of signal S1-S2 and an output from 1H delay line 510 in response to a line selecting pulse, thereby outputting color differential signal CR (=2R-G) and color differential signal CB (=2B-G) for each line, and a matrix circuit 514 for performing a prescribed linear operation to the output from selecting circuit 512 and signal S1+S2, in other words luminance signal CL which it receives and outputting separated three primary color signals R, G and B.
FIG. 22 is a diagram schematically showing the relation between signals output from CCD 10 and sample-hold pulses SP1 and SP2 for sampling and holding the signals.
Referring to FIGS. 21 and 22, the operation of the conventional color separation circuit 500 shown in FIG. 21 will be described.
Although a signal processing in an even-numbered field will be described, the basic operation is entirely the same for an odd-numbered field except that rows subjected to a signal addition are shifted.
Assuming that CCD outputs from n-th line in the even-numbered field have been sampled and held in response to sample-hold pulses SP1 and SP2, signals S1 and S2 output from sample-hold circuits 502 and 504 will be considered.
Referring to FIG. 22, signal S1 sampled and held in response to sampling pulse SP1 is a sum signal of signal Ye and signal Mg for the n-th line. Signal S2 sampled and held in response to sampling pulse SP2 is a sum signal of signal Cy and signal G.
The sum and difference between signals S1 and S2 are as follows: EQU S1+S2=Ye+Mg+Cy+G=2R+3G+2B (8) EQU S1-S2=(Ye+Mg)-(Cy+G)=2R-G (9)
Signal S1 produced by sampling and holding the output signal from the n+1st line in the CCD in the even-numbered field in response to sampling pulse SP1 is a sum signal of signal Ye and signal G, and signal sampled and held in response to sampling pulse SP2 is a sum signal of signal Cy and signal Mg.
A differential difference signal between signals S1 and S2 is therefore represented as follows:
S1-S2=(G+Ye)-(Mg+Cy)=-(2B-G) (10)
then for 2R-G=CR, -(2B-G)=CB, and 2R+3G+2B=CL, signal CR and signal CB can be alternately obtained for each line by producing differential signal S1-S2.
In order to produce a color signal for each single scanning line, luminance signal CL, color differential signals CR and CB are necessary for the scanning line.
In color separation circuit 500, to a particular scanning line such as the n+1st line, color signals are produced/separated in the n+1st line by using color differential signal CB obtained from the n+1st line and color differential signal CR obtained from the n-th line.
More specifically, signal S1-S2 (CR, CB, . . . ) output from subtracter circuit 508 and a signal (CB, CR, . . . ) obtained by passing the signal through 1H delay line 510 are input to selecting circuit 512 switched in response to a line selecting pulse, so that color differential signal S1-S2 one scanning line before may be used as a color differential signal for the scanning line which is currently being read.
More specifically, selecting circuit 512 directly outputs signal S1-S2 corresponding to the presently read scanning line to matrix circuit 514 as color differential signal CR if the signal is signal CR, and at the same time outputs a color differential signal one scanning line before which is output from 1H delay line 510 as signal CB to matrix circuit 514.
Conversely, signal from 1H delay line 510 is output as color differential signal CR and signal S1-S2 from subtracter circuit 508 is output as color differential signal CB to matrix circuit 514 if color differential signal S1-S2 read out from the present scanning line corresponds to signal CB.
Matrix circuit 514 outputs three primary color signals R, G and B according to the following transforms from three received signals, luminance signal CL and color differential signals CR and CB. EQU G=(2CL-2CR-2CB)/10 (11) EQU R=(CL+4CR-CB)/10 (12) EQU B=(CL-CR+4CB)/10 (13)
Signals S1 and S2 result from sampling and holding the output signal of the CCD in response to sampling frequency fn. FIG. 23 shows the frequency characteristic of signals S1 and S2.
Signals S1 and S2 result from sampling and holding, the response of which gradually decreases from the low frequency region and reaches 0 at frequency fn/2.
Luminance signal CL, color differential signals CR and CB and RGB signal resulting from these signals have similar frequency characteristic as FIG. 23.
The horizontal resolution of the RGB signal is therefore H/2. The vertical resolution within a single field is V/4 for signals CR and CB, and V/2 for luminance signal CL. For the RGB signal resulting from these signals, the vertical resolution is about V/4.
As described above, the resolution corresponding to only 1/2 to 1/4 the number of effective pixels can be obtained for RGB.